Journal: Genome Biology

Article Title: A systematic evaluation of single-cell RNA-sequencing imputation methods

doi: 10.1186/s13059-020-02132-x

Figure Lengend Snippet: Impact of imputation methods on differential expression analysis. For each imputation method, we performed three gene-level analyses. a Schematic view of evaluating differentially expressed genes (DEGs) using the overlap between bulk RNA-seq and scRNA-seq. b – d Proportion of overlap between bulk and single-cell DEGs identified using either MAST ( x -axis) or Wilcoxon rank-sum test ( y -axis). Note that “cl” in the names of datasets means “cell line.” e Schematic view of a null DE analysis. f – h Number of false positive DEGs averaging across all settings identified by MAST ( x -axis) or Wilcoxon rank-sum test ( y -axis) in null differential analyses. i Heatmap of area under a receiver operating characteristic (ROC) curve values when using the expression level of a marker gene (e.g., CD19) to predict a cell type (e.g., B cell or not) using UMI-based sorted PBMC cell types. For some imputation methods, no imputed values were returned. They are denoted as “ImputationFail”. j , k Using a UMI-based scRNA-seq dataset from cell lines ( sc_10x_5cl ), a heatmap showing the percentage of the overlap between bulk and single-cell DEGs identified using MAST stratified by genes with high (top 10%) or low (bottom 10%) log-fold changes. The color bar on the last column shows the mean overlap across all comparison for each method. If MAST failed to identify DEGs from the imputed profiles of any method in any dataset, we denoted it as “DifferentialFail.” Please refer to Additional file : Figure S5 for the Wilcoxon results

Article Snippet: Single-cell profiles from five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ) were combined into one count matrix which was used as input to each imputation method.

Techniques: Quantitative Proteomics, RNA Sequencing, Expressing, Marker, Comparison

Journal: Genome Biology

Article Title: A systematic evaluation of single-cell RNA-sequencing imputation methods

doi: 10.1186/s13059-020-02132-x

Figure Lengend Snippet: Impact of imputation methods on unsupervised clustering analysis. a Heatmap of four performance metrics—entropy of cluster accuracy ( H acc ), entropy of cluster purity ( H pur ), adjusted Rand index (ARI), and median Silhouette index—averaged across seven datasets from CellBench . Each metric shows the average performance across 7 datasets in CellBench. To compare imputation methods across metrics, the metrics were re-scaled to between 0 and 1 and the order of H acc and H pur were flipped to where a higher standardized score translates to better performance. Imputation methods (rows) are ranked by the average performance between the mean of the first three metrics ( H acc , H pur and ARI) and the fourth metric ( medianSil ). b Dimension reduction results after applying PCA to the sc_celseq2_5cl_p1 data with no imputation (left) and with imputation using MAGIC (right). The colors are the true group labels. c Overall score (or average of the four performance metrics) for Louvain clustering ( x -axis) and k -means clustering ( y -axis). d – f Same as a – c except using the scRNA-seq dataset of ten sorted peripheral blood mononuclear cell (PBMC) cell types from 10x Genomics ( PBMC_10x_tissue dataset). White areas with black outline in d indicate that the imputation methods did not return output after 72 h. Also, e uses UMAP components instead of principal components. Please refer to Additional file : Figures S6, S7, S9 for Louvain clustering results and metrics in each dataset and Additional file : Figure S8 for UMAPs of other methods

Article Snippet: Single-cell profiles from five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ) were combined into one count matrix which was used as input to each imputation method.

Techniques:

Journal: bioRxiv

Article Title: A Systematic Evaluation of Single-cell RNA-sequencing Imputation Methods

doi: 10.1101/2020.01.29.925974

Figure Lengend Snippet: For each imputation method, we performed three gene-level analyses. (A) Schematic view of evaluating differentially expressed genes (DEGs) using the overlap between bulk RNA-seq and scRNA-seq. (B-D) Proportion of overlap between bulk and single-cell DEGs identified using either MAST (x-axis) or Wilcoxon rank-sum test (y-axis). Note that ‘cl’ in the names of datasets means ‘cellline’. (E) Schematic of a null differential expression analysis. (F-H) Number of false positive DEGs identified using MAST (x-axis) or Wilcoxon rank-sum test (y-axis). (I) Heatmap of area under a receiver operating characteristic (ROC) curve values when using the expression level of a marker gene (e.g. CD19) to predict a cell type (e.g. B cell or not) using UMI-based sorted PBMC cell types. For some imputation methods, no imputed values were returned. They are denoted as “ImputationFail”. (J-K) Using a UMI-based scRNA-seq dataset from cell lines ( sc_10x_5cl ), a heatmap showing the percentage of the overlap between bulk and single-cell DEGs identified using MAST stratified by genes with high (top 10%) or low (bottom 10%) log-fold changes. The color bar on the last column shows the mean overlap across all comparison for each method. If MAST failed to identify DEGs from the imputed profiles of any method in any dataset, we denoted it as “DifferentialFail”.

Article Snippet: A similar procedure was applied to the five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ).

Techniques: RNA Sequencing, Quantitative Proteomics, Expressing, Marker, Comparison

Journal: bioRxiv

Article Title: A Systematic Evaluation of Single-cell RNA-sequencing Imputation Methods

doi: 10.1101/2020.01.29.925974

Figure Lengend Snippet: (A) Schematic of evaluating differentially expressed genes (DEGs) using the overlap between bulk RNA-seq and scRNA-seq – also shown in . Using the pairs of cell lines in the sc_10x_5cl dataset, ENCODE_fluidigm_5cl dataset, and pairs of cell types in the bone marrow tissue from the HCA_10x_tissue dataset, we show heatmaps of proportion of overlap between bulk and single-cell DEGs identified using (B, D, F) MAST and (C, E, G) Wilcoxon-rank-sum test (abbreviated as Wilcoxon) for differential expression, respectively. (H) Schematic of a null differential expression analysis by randomly partitioning cells from the same cell type into two groups – also shown in . Using the 293T cells from the 10x_293t_jurkat dataset, the GM12878 cells from the ENCODE_fluidigm_5cl dataset, and bone marrow cells from the HCA_10x_tissue dataset, the number of false positive DEGs identified using (I, K, M) MAST and (J, L, N) Wilcoxon, respectively. The x-axis in Figures (I-N) describe the number of cells in each group (e.g. 10 sampled cells in group 1 and 10 sampled cells in group 2) when applying a method to identify differentially expressed genes. White areas with black outline indicate that the imputation methods did not return output after 72 hours and areas with grey outline indicate that either MAST or Wilcoxon failed to return results.

Article Snippet: A similar procedure was applied to the five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ).

Techniques: RNA Sequencing, Quantitative Proteomics

Journal: bioRxiv

Article Title: A Systematic Evaluation of Single-cell RNA-sequencing Imputation Methods

doi: 10.1101/2020.01.29.925974

Figure Lengend Snippet: For each cell line in the sc_10x_5cl dataset, we calculated the gene-specific standard deviation across cells with no imputation ( no_imp ) and with imputation.

Article Snippet: A similar procedure was applied to the five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ).

Techniques: Standard Deviation

Journal: bioRxiv

Article Title: A Systematic Evaluation of Single-cell RNA-sequencing Imputation Methods

doi: 10.1101/2020.01.29.925974

Figure Lengend Snippet: The pairs of cell lines from the sc_10x_5cl dataset compared below are (A) H2228 ( N =758) vs H838 ( N =876) – more balanced group sizes and (B) A549 ( N =1256) vs H1975 ( N =440) – less balanced group sizes. For each pair of cell lines, we applied MAST and extracted and plot the following information: the distribution of estimated log-fold changes (‘coefficients’) (top left), the distribution of standard errors for the coefficients (top right), the distribution of test statistics (z-score output from MAST, bottom left), and distribution of log-transformed p -values (bottom right).

Article Snippet: A similar procedure was applied to the five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ).

Techniques: Transformation Assay

Journal: bioRxiv

Article Title: A Systematic Evaluation of Single-cell RNA-sequencing Imputation Methods

doi: 10.1101/2020.01.29.925974

Figure Lengend Snippet: (A) Heatmap of four performance metrics – entropy of cluster accuracy ( H acc ), entropy of cluster purity ( H pur ), adjusted Rand index (ARI), and median Silhouette index – averaged across seven datasets from CellBench . Each metric shows the average performance across 7 datasets in CellBench. To compare imputation methods across metrics, the metrics were re-scaled to between 0 and 1 and the order of H acc and H pur were flipped to where a higher standardized score translates to better performance. Imputation methods (rows) are ranked by the average performance across all four metrics. (B) Dimension reduction results after applying PCA to the sc_celseq2_5cl_p1 data with no imputation (left) and with imputation using MAGIC (right). The colors are the true group labels. (C) Overall score (or average of the four performance metrics) for Louvain clustering (x-axis) and k -means clustering (y-axis). (D-F) Same as (A-C) except using the scRNA-seq dataset of ten sorted peripheral blood mononuclear cell (PBMC) cell types from 10x Genomics ( PBMC_10x_tissue dataset). White areas with black outline in (D) indicate that the imputation methods did not return output after 72 hours. Also, Figure (E) uses UMAP components instead of principal components.

Article Snippet: A similar procedure was applied to the five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ).

Techniques:

Journal: bioRxiv

Article Title: A Systematic Evaluation of Single-cell RNA-sequencing Imputation Methods

doi: 10.1101/2020.01.29.925974

Figure Lengend Snippet: (A) Heatmap of four performance metrics – entropy of cluster accuracy ( H acc ), entropy of cluster purity ( H pur ), adjusted Rand index (ARI), and median Silhouette index – averaged across seven datasets. To compare imputation methods across metrics, the metrics were re-scaled to be between 0 and 1 and the order of H acc and H pur were flipped to where a higher standardized score translates to better performance. Imputation methods (rows) are ranked by the average across all four metrics. (B) Dimension reduction results after applying PCA to the sc_celseq2_5cl_p1 data with no imputation (left) and with imputation using MAGIC (right). The colors are the true group labels. (C) Overall score (or average of the four performance metrics) for Louvain clustering (x-axis) and k -means clustering (y-axis). (D-G) Heatmaps of the individual performance metrics (D) H acc , (E) H pur , (F) ARI and (G) the median of Silhouette of each imputation method for each CellBench dataset. The white boxes with black lines represent cases in which no output was returned from the imputation method after 72 hours.

Article Snippet: A similar procedure was applied to the five Fluidigm-based ENCODE cell lines ( ENCODE_fluidigm_5cl ).

Techniques: